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WHAT IS CLAIMED IS: 



1 1. 



A system, comprising: 



2 a computer configured to determine a position and shape of an object of 

3 interest from video images and characterize activity of said object of interest 

4 based on analysis of changes in said position and said shape over time. 



1 2. The system of claim 1, further composing: 



a video camera coupled to said computer for providing said video images. 
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1 3. The system of claim 2, furthefr comprising: 

2 a video digitization unit cmiple to said video camera and said computer 

3 for converting said video image^provided by said video camera from analog to 

4 digital format. 



1 4. The system of claim 3, further comprising: 



a storage/retrieval unit coupled to said video digitization unit, said video 
camera, and said corqputer, for storing said video images and standard object 
video images. 
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1 5. The system of f^X^m. 1, wherein said computer includes an object 

2 identification and se^^gattoTTmodule receiving said video images. 




1 6. The system of claim 5, wherein said objtct identification and segregation 

2 module operates using a background suJ>{raction algorithm in which a plurality 

3 of said video images are grouped into a set, a standard deviation map of the set 

4 of video images is created, a bounding box where a variation is greater than a 

5 predetermined threshold is/emove from said set of video images, and the set of 

6 images less said bounding boxes is averaged to produce a background image. 

1 7. The system of claim 6, wherein said comput/r further includes a behavior 

2 identification module for characterizing activit/ of said object, said behavior 

3 identification module being coupled to said ohgect identification and segregation 

4 module. / 

1 8. The system of claim 7, wherein said computer fiirther includes an object 

2 tracking module for tracking said object fi-om one fi'ame of said video images to 

3 another fi'ame, and an object shm)e and location change classifier for classifying 
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the activity of said object, coupled to each other, iaid object identification and 
segregation module, and said behavior identificaltion module. 



9. The system of claim 8, wherein said computer further includes a standard 
object behavior storage module that stores information about known behavior of 
a predetermined standard object for comparing the activity of said object, said 
standard object behavior storage modpe being coupled to said behavior 
identification module, and a standard object classifier module coupled to said 
standard object behavior module. 



1 0. The system of claim 5, wherein said computer further includes a standard 
object behavior storage module that stores information about known behavior of 
a predetermined standard objector comparing the activity of said object, said 
standard object behavior storage module being coupled to said behavior 
identification module. 



1 1 . The system of claim 1, wherein said object is a living object. 



12. The system of claim 1, wherein said object is an animal. 
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1 13. The system of claim 1 , wherein said object is a mouse. 



1 14. The system of claim lyWherein said object is a human. 



1 15. The system /)f claim 1, wherein said object is a man made machine. 

1 16. A method of determining and characterizing activity of an object using 

2 computer processing of video images, comprising the steps of: 

3 detecting a foreground objeK;t of interest in said video images; 

4 tracking changes to said foreground object over a plurality of said video 

5 images; 

6 identifying and classifying said ch^ges to said foreground object; and 

7 characterizing said activity of said for^ound object based on comparison 

8 to activity of a standard object. 




1 1 7. The method of clSi^ 1^, wl^erein said step of characterizing said activity 

2 includes the steps of: ' 

3 describing a sequence of postures as behavior primitives; and 
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aggregating behavior wfmiitives into actual behavior over a range of 



images. 
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1 18. The method of claim 16, wherein said foregp-ound object detection 

2 includes the step of generating a background image frpm an average of a set of 

3 individual frames of said video images. 



1 19. The method of claim 18, wherein said step of generating a background 

2 image includes the step of determining variation in intensity of pixels within said 

$ I 

3 individual frames to identify a region where said foreground object is located. 



1 20. The method of claim 19, wherein said step of generating a background 

2 image further includes the step of using non-variant pixels of the video images 

3 to generate said background image. 



1 21. The method of claim 20, wherein said step of generating a background 

2 image is performed periodically to collect for changes in background objects and 

3 small movements of a camera capturing said video images. 



1 22. The method of claim 16/ wherein said detecting a foreground object 
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2 includes using a background subtraction method compriafng the steps of: 

3 multiply frames in a neighborhood of current image; 

4 apply a lenient threshold on a difference between a current image and a 

5 background so as to determine a broad region of interest; 

6 classify by intensity various pixels in said region of interest to obtain said 

7 foreground object; and / 

8 apply edge information to refine con/tours of said foreground object 

9 image. / 

1 23. The method of claim 16, wherein said step of detecting said foreground 

2 includes the step of manual identification of foreground objects to be tracked and 

3 characterized. / 




1 24. The method of claim 17, wherein said posture determination and 

2 description includes using stati/ical and contour-based shape information. 
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25. The method of claim/24, wherein said step of identifying and classifying 

/ 

changes to said foreground object includes using statistical shape information 

/ 



selected from the group consisting of: 
area of the foreground object; 
centroid of the/foreground object; 
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6 bounding box and its aspect ratio of the fo/eground object; 

7 eccentricity of the foreground object; anc 

8 a directional orientation of the foregrdund object relative to an axis as 

9 generated with a Principal Component Analysis. 

1 26. The method of claim 24, wherein said step of identifying and classifying 

2 changes to said foreground object uses contour-based shape information selected 

3 from the group consisting of b-spline representation, convex hull representation, 

4 and comer points. 



1 27. The method of claim 24, wherein said step of identifying and classifying 

/ 

2 changes to said foreground object includes identifying a set of model postures and 



3 their description information, said set of model postures including horizontal 

/ 

4 posture, vertical posture, eating posture, or sleeping posture. 



/ 

1 28. The method of claim 27, wherein said step of identifying and classifying 



2 changes to said foreground object includes classifying the statistical and contour- 

3 based shape information from a current image to assign a best-matched posture. 
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1 29. The method of claim /?, wherein the said step of describing said behavior 



2 primitives includes the 

3 images. 




ifying patterns of postures over a sequence of 



1 30. The method of claim 29, wherein said step of describing said behavior 

2 primitives step further includes/ the step of analyzing temporal information 

3 selected from the group consistftng of direction and magnitude of movement of 

4 the centroid, increase and deorease of the eccentricity, increase and decrease of 

5 the area, increase and decrease of the aspect ratio of the bounding box, change in 

6 the b-spline representation points, change in the convex hull points, and direction 

7 and magnitude of comer points. 
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31. The method of claim 29, whereii/the step of describing said behavior 
primitives step includes behavior of a standard object such as stationary, moving 
for left to right and vice versa,/standing up, and falling down. 




1 32. The method of claim/29, wherein the step of describing said behavior 

2 primitives step includes a step for providing a means for entering user defined 
customized behavior primitives. 
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1 33. The method of claim 17, wherein the said step pf letermining actual 

2 behavior by aggregating behavior primitives inclmiSs the step of analyzing 

3 temporal ordering of the primitives, such as using information about a transition 

4 from a previous behavior primitive to a nex/behavior primitive. 



1 34. The method of claim 33, v^herein said temporal analysis is a time-series 

2 analysis such as Hidden Markoy Model (HMMs). 



1 35. The method of /laim 33, wherein the said step of determining actual 

2 behavior includes identifying actual behavior selected from a group consisting of 

3 sleeping, eating, roaming around, grooming, and climbing. 



1 36. A method for backgrourid^subtraction of a video image, comprising the 

2 steps of: 

3 grouping a number of images into a set of video images; 

4 creating a standard deviation mapW the grouped images; 

5 removing a bounding box area of said image where variation is above a 

6 predetermined threshold to create a partial image; and 
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7 combining said partial image with an existing set of partial images by 

8 averaging the set of images to generate a complete background image deplete of 

9 a desired foreground object. \ 

1 37. The method of claim 36, futmcr comprising the step of subtracting said 

2 complete background image from a current image so as to obtain said desired 

3 foreground object. X 

1 38. The motnod of claim 36, wherein said steps are repeated periodically to 

2 update sauf complete background image. 

1 39. A system, compriiing: 

2 a computer configured to detect and characterize at least a single behavior 

3 of an object of interest MSSff^an/movement of said object, using video image 

4 analysis. \A // 

1 40. The system of claim 39, wherein said object is an animal and said 

2 behavior is detecting when said animal is freezing or a touch or sniff of a 

3 particular item. I 
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41. The system of claim 39, wherein said object is an animal and said 
detecting and characterizing said behavior is determined by comparing behavior 
of said animal against a predetermined norm. 



42. The systerm ofxdaii^^9, wherein said object is an animal and 
characterizing said bma^^ lytietermined by analyzing a daily pattern of said 
object against a statMrdal norm so as to detect effects of drugs or genetic 
manipulations on sai i anima. 



60 



